With the evolution of large language models, traditional Transformer models become computationally demanding for lengthy sequences due to the quadratic growth in computation with respect to the sequence length. Mamba, emerging as a groundbreaking architecture in the field of generative AI, demonstrates remarkable proficiency in handling elongated sequences with reduced computational and memory complexity. Nevertheless, the existing training framework of Mamba presents inefficiency with variable-length sequence inputs. Either single-sequence training results in low GPU utilization, or batched processing of variable-length sequences to a maximum length incurs considerable memory and computational overhead. To address this problem, we analyze the performance of bottleneck operators in Mamba under diverse tensor shapes and proposed PackMamba, a high-throughput Mamba that efficiently handles variable-length sequences. Diving deep into state-space models (SSMs), we modify the parallel operators to avoid passing information between individual sequences while maintaining high performance. Experimental results on an NVIDIA A100 GPU demonstrate throughput exceeding the baseline single-sequence processing scheme: 3.06x speedup on the 1.4B model and 2.62x on the 2.8B model.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

While some view videogames as a waste of time, researchers have found that spending time in these virtual worlds can actually enhance your perception and attention skills. People who play between one and five hours a week are able to process visual information five percent more accurately than those who don't play at all, finds a new study. These findings suggest that those who play video games are faster and more efficient at processing rapidly-presented stimuli. People who play between 1 and 5 hours a week are able to process visual information 5% more accurately than those who don't play at all, finds a new study from psychologists at Nottingham Trent University Nottingham Trent University asked 43 participants to perform observation tasks presented on a screen. The first asked participants to identify a white letter within the stream of black letters, and the other the letter'T' in one of four orientations, rotated by 0, 90, 180, and 270 and observers attempted to discriminate between these orientations The team found that video gamers were able to perform this dual task on average 5% more accurately than non-gamers, suggesting faster and more efficient processing of rapidly-presented stimuli.